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Classifying gesture or movements nowadays become a demanding business 
as the technologies of sensor rose. This has enchanted many researchers 
to actively investigated widely within the area of computer vision. 
Rehabilitation exercises is one of the most popular gestures or movements 
that being worked by the researchers nowadays. Rehab session usually 
involves experts that monitored the patients but lacking the experts itself 
made the session become longer and unproductive. This works adopted 
a dataset from UI-PRMD that assembled from 10 rehabilitation movements. 
The data has been encoded into spike trains for spike patterns analysis. 
Next, we tend to train the spike trains into Spiking Neural Networks 
and resulting into a promising result. However, in future, this method 
will be tested with other data to validate the performance, also to enhance 
the success rate of the accuracy. 
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1. INTRODUCTION 

Physiotherapy, also known as physical therapy, is an ongoing treatment for those who struggling 
with serious illness such as stroke, Parkinson’s, post-surgeries, etc. This therapy is extremely helpful in order 
for the patients to cope with their daily living tasks due to their movement’s impairment. The assessment 
movements usually will be prescribed according to the patient’s condition within their own pace as 
the movement for a patient may not be equally adequate for others [1]. Furthermore, the patients need 
to be monitored while performing the movements so that they executed correct movements and their 
progression can be tracked by medical experts. However, lack of experts makes physiotherapy session 
delayed and caused discomfort to the patients and the caregivers as well. Hence, technologies take places 
in assisting the experts conducting the session. Physiotherapy has caught interests of many researchers 
in regard to machine learning approaches. Different machine learning algorithms has been utilized 
for recognizing different types of physiotherapy movements, also recognizing different parts of the body. 

Recently, neural networks architecture have been a favour for its flexibility and compatibility 
in a wide range of machine learning applications including computer vision [2]. In the last few decades, 
the 3rd generation of neural networks, spiking neural networks (SNNs) has been introduced. SNNs are said 
to be a promising paradigm [3] as it is practically efficient of modelling complex information as they 
are adequate to represent and integrate different information dimensions (time, space, frequency, phase). 
Their flexibility and self-organizing manner make it effortless when they are dealing with large volumes 
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of data as they are using information representation as trains of spikes [4]. Thus, SNNs intent to span the gap 
between neuroscience and machine learning which leads in boosting a number of researches with the focus 
biologically motivated approaches for pattern recognition [5, 6]. 

This paper employs University of Idaho-Physical Rehabilitation Movements Data Set (UI-PRMD) 
[7] which is a data set of movements performed by patients in physical therapy and rehabilitation programs 
to classify the types of the movements. This paper proposed spiking neural network (SNN) model to analyse 
the spike which highly informative for actions recognition to classify the types of the movements. This paper 
is structured as follows: Section 2 summarizes related works in classification using UI-PRMD. Section 3 
explained the proposed methodology in analysing spike train and classifying by Spiking Neural Network. 
In section 4, presents experimental results and discussions of the proposed approaches. Finally, conclusions 
are drawn in Section 5. 


2. PREVIOUS WORK 

Machine learning has been said to be potentially guiding rehabilitation sessions especially in-home 
rehab sessions as it can improved the ability of recognition and classification. These techniques nowadays 
are being used extensively in biomedical application [8], for example in recognizing body parts and the types 
of movements. Previously, Zhu et. al [9] applied K-Nearest Neighbors (KNN) algorithm on rehabilitation 
prediction and resulting that KNN made significantly better prediction that the Activity of daily living 
clinical assessment protocol, ADLCAP which being used within the health assessment information system 
in Canada . Next, they employs Support Vector Machine in [10] to improve upon KNN algorithms as SVM 
may give more accurate predictions. However, turns out that SVM does not statistically improved over KNN. 
Muniz et. al [11] compare three different models, SVM, Probabilistic Neural Network (PNN), and Logistic 
Regression (LR) for discriminating between normal and Parkinson disease persons by monitoring their 
walking postures. PNN seems performed better than SVM and LR as PNN showed high performance indexes 
in classifying ground reaction force of normal and Parkinson subjects. While Patsadu et. al [12] compared 
four approaches for pose classification, Backpropagation Neural Network (BNN), SVM, Decision Tree, 
and Naive Bayes. This resulting in BNN and SVM protrude over the other approaches. On the contrary, 
Neural Networks (NN) and Support Vector Machine (SVM) are also commonly used for poses and motion 
recognition. Sudani et. al [13] employs SVM to identify a person state whether they in a normal or anomaly 
movement for fall detection while doing home-based rehabilitation exercises. On the other hand, 
there are limitations in speed and size for SVM classification. Ciresan et al. used Convolutional Neural 
Networks (CNN) [14], while Toshev et al. implement Deep Neural Networks (DNN) to recognize human 
poses and activities [15]. Du et. al divided human skeleton into five segments and used each of the parts 
to train a hierarchical recurrent neural networks [16]. 

DNNs are said as a historically brain-inspired but the fundamental are differences in structure, 
neural computations, and learning rule compare to the brain. The most important difference is the way 
of the information propagates between their units. Neurons communicate to each other in a neural circuit by 
sending spike trains which sparse in time so that they have high information content. Spiking Neural 
Networks (SNNs) are thus more biologically and realistic than DNNs. SNNs also are more hardware friendly 
and energy-efficient. SNNs shown a promising performance in number of pattern recognition tasks. 
The architecture consists of spiking neurons where the spike trains communicate with each other through 
synaptic connections to stimulate the corresponding system. Within the learning from data, the synaptic 
connections are modified to reflect more precisely the timing of the data from the sensory inputs. 

Presently, artificial sensors do not generate spikes as the primary signal instead they are providing 
us with an electrical analog or digital output encoding its measured value. Thus, various spike encoding 
method being introduced [17]. SNN has been widely implemented in various area such speech recognition 
technology as speech signals were converted into spike trains signatures where they are able to differentiate 
the speech signals that representing different words [18]. In [19], they equipped spike trains for recognition 
of object and handwritten character and classified with SVM. Next, [20, 21] converting analog signals 
of SA-I mechanoreceptors in human skin into spike trains by an Izhikevich’model and achieved to classify 
naturalistic textures with an accuracy of 97%. Hence, they found that SNN exhibit good reliability 
with strong generalization power of spiking neurons. 


3. RESEARCH METHODOLOGY 
3.1. Spike train 

SNN represents a special class of artificial neural network (ANN) [22] where ANN have been well 
developed and had lot of successful application. SNN, as the third generation of neural network [19] also 
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to be said as having the potential on solving problem in bio-related stimulation. This neuron models 
propagates by sequences of spikes or action potentials which conveying information to each other 
in a membrane potential. The sequences of spikes, or also to be called spike trains, consisting of n spikes 
occurring at time ti can be represented mathematically as: 

pit) = E? =1 5 (t - ti) (1) 

Where 5 is the Dirac delta function. In order to quantifying the average responses encoded by a single spike 
train is to simply add up the spikes over sometime interval T: 

p(x)dT (2) 

This is occasionally called the “firing rate” or “spike count rate”. Next, we define an average firing rate over 
the trials as below: 

^J o r (p(r))dT (3) 

Where () denotes the average over trials. We also define a firing rate with finer temporal resolution 
as defining a firing rate r(t) as: 

rit) = ^ J t t+At (p(r))dr (4) 

Which also can be used to define Post-Stimulus Time Histogram (PSTH). However, PSTH always defined 
by depending on where the time bins are placed. Hence, another way to define a firing rate is: 

rit) = 1™^ coir) pit - r)dr (5) 

Where coir) is called the filter kernel. Nonetheless, a neuron does not have access to the latter information 
a sit is not causal. Thus, a better model conjointly with causal filter such as: 

co it) = [a 2 rexp(—crr)] + (6) 

Where [x] + means x for x > 0, and 0 otherwise. Consequently, we can fit the firing rate to a simple function 
of the stimulus attributes. 

3.2. Spiking neural networks 

Spiking neurons and synapse are basically the units of an SNN that interconnecting within them. 
We use the leaking integrate and fire (LIF) [23] models of neurons for the computational tractability where 
the evolution of the membrane potential, V m (t) is describe by: 

c = gL(V m (t) - E l ) + 7(t) (7) 

Here lit) is the total input current, E L is the resting potential, and C (300 pF) and gL (30 nS) model the 
membrane capacitance and leak conductance, respectively [24]. Once the membrane potential crosses 
a threshold (V m (t) > V T ), it is reset to its resting value E L and remains at that value till the neuron comes out 
of its refractory period {t re f = 3 ms ). The synapse, with weight w k t connecting input neuron k 
to output neuron Z, transforms the incoming spikes (arriving at times t k , t k , . . .) into a post-synaptic current 
(/ fc Z ), based on the following transformation, 

C fc (t) = <s(t- 4) *{e T 1 - e (8) 

I = w kJ x c k (t) (9) 

Here, the summed 8 function represents the incoming spike train and the double decaying exponentials 
with r 1 = 5 ms andr 2 = 1.25 ms represent the synaptic kernel [25]. 
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4. RESULTS AND ANALYSIS 

4.1. Experimental setting 

This study tends to propose a classification method for spike train analysis by implementing 
three-layered spiking neural network. We proposed SNN classification model and documented 
the achievements. The experiment was carried out using normal CPU and scripted in Matlab & Simulink 
library. The computational time is only 30s per 100 sample. Figure 1 illustrated the general framework of 
proposed algorithm. Raw data has been parted into 100 for each sample and being generated into firing rates 
to contrast the movements’ correctness. Next, we processed the data into spike trains for classification input. 
We implement the spike patterns into SNN algorithm for movement classification and tabulate 
with confusion matrix for validation. Details of each part will be explained in the subtopic below. 
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Figure 1. General framework of proposed algorithm 


4.2. Dataset 

This paper employs UI-PRMD dataset [7] which consists 10 rehabilitation movements performed 
by 10 healthy individuals. Each person performed each movement repetitively 10 times in front of two 
sensory systems for motion capturing which are Vicon optical tracker and a Kinect camera. The data 
is collected as positions and angles of the body joints in skeletal models provided by the sensors itself. 
However, this paper will be focused on Kinect camera angles data with 5 selected rehabilitation movements; 
Deep Squat, Hurdle Step, Inline Lunge, Sit to Stand, and Standing Shoulder Extension. The results 
are assessed and discussed in two parts; Spike Train Analysis and SNN Classification. 

4.3. Spike train analysis 

In spike train analysis, data has been parted into 100 frames for each sample and encoded into firing 
rate and spike train. Firing rate is basically related to the number of spikes generated by a neuron per unit 
of time. However, in this study, we describe each neuron’s firing rate as a function of direction of stimulus 
motion. Figure 2 shows the comparison of the correctness for the movements which firing spikes generated 
along 0° till 360° orientation direction. The frequencies of firing rates of subjects might be varied as they 
mostly have different body mass index, but the pattern of the firing rates should be the same to consider 
the correctness. The incorrect movements might be assembled by incorrect angles of body joints; hence they 
did not achieve the target angle for perfect movements. 

Spike train is built up from a sequence of recorded time at which a neuron fires an action potential. 
Accumulation of the spike trains in this experiment, leads to a unique pattern for each movement. 
From Figure 3, each of the movement being recognized has their unique spike patterns which reflect some 
aspects of neural functioning such as relating neural activity to stimuli, finding a repetitive pattern in a motor 
discharge and functional interactions between the neurons. Deep Squat and Inline Lunges pattern can be 
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defined with similar patterns as physically the movement itself homogenous to each other with the up 
and down movement direction. 




(c) 

Figure 2. Example of Comparison of movements’correctness for, (a) Deep squat, 
(b) Hurdle step, (c) Inline lunges 





Time 




Time 


(a) 


(b) 


(c) 


Figure 3. Example of spike train patterns for, (a) Deep squat, (b) Hurdle step, (c) Inline lunges 


4.4. SNN classification 

In this paper, SNN has been adopted to classify spike trains pattern according to the type 
of the movements performed. The proposed study is divided into two phases; training and testing. During 
the training phase, SNN has been trained offline to predict and classify the type of movement from the spike 
train patterns. For the testing phase, few spike trains were chosen randomly to validate the accuracy 
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of the SNN classification. After stimulating the network with spike trains, each neuron starts to specialize 
to respond to a certain pattern. The ratio of the test and training data is 4:1 respectively. 

Figure 4 illustrated the accuracy of the classification for the movements based on spike trains 
analysis classified by SNN. It can be seen from the confusion matrix, SNN achieved 0.71 accuracy 
and clearly illustrated that the similarity values between the spike’s patterns are little bit scattered 
and Standing Shoulder Extension has the lowest value as it diverged into Deep Squat by 0.29. 
This discrepancy occurs because of the locational error for the spatial features. Moreover, the patterns 
of the spike trains are similar to each other so that SNN can’t differentiate and classify them accurately. 
However, in future, we intend to increase the range of the data so that the accuracy might increasing as well. 



Deep Squat Hurdle Step Inline Lunges Sit to Stand Shoulder Extension 


Figure 4. Confusion Matrix for SNN classification accuracy of rehabilitation exercises movement 


5. CONCLUSION 

In this paper, an approach of spiking neural network classification in spike train analysis was 
proposed. We proposed a simple three-layer spiking neural network that performs the classification 
of the spike trains to classify the similarity values with the patterns. Our approach achieves 71% of accuracy 
which as expected. Thus, this approach is suitable for rehabilitation to be implemented in a stand-alone 
rehabilitation monitoring session. In future, this method will be tested with other data to validate 
the performance, also to enhance the success rate of the accuracy. However, we need to revise the algorithm s 
in achieving desired results. Hence, further filtering pre-processing will be implemented to strengthen 
the proposed method. We will further compare with the other approaches such as SVM, ANN, etc in terms 
of the classification accuracy. As a conclusion, this proposed method works as our expectation, however, 
need to be improve for further achievement. 
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